Papers by Joshua Ong Jun Leang
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Mathematical reasoning remains a significant challenge for large language models (LLMs), despite advances in prompting techniques such as Chain-of-Thought (CoT). |
| Approach: | They propose a framework that enhances reasoning through two stages: Symbolic Conversion and Reasoning Execution. |
| Outcome: | The proposed framework outperforms traditional CoT on six out of seven benchmarks across four LLMs. |
PiCSAR: Probabilistic Confidence Selection and Ranking for Reasoning Chains (2026.findings-acl)
Copied to clipboard
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema, Sohee Yang, Wai-Chung Kwan, Xuanli He, Wenda Li, Pasquale Minervini, Eleonora Giunchiglia, Shay B Cohen
| Challenge: | Recent studies show that large reasoning models (LLMs) achieve strong performance on complex reasoning tasks. |
| Approach: | They propose a method that scores each candidate generation using the joint log-likelihood of the reasoning and final answer. |
| Outcome: | The proposed method outperforms baselines with 2x fewer samples in 20 out of 25 comparisons. |
Are We Done with MMLU? (2025.naacl-long)
Copied to clipboard
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini
| Challenge: | MMLU is widely adopted but its ground truth errors obscure the true capabilities of LLMs. |
| Approach: | They propose a framework for identifying dataset errors using a novel error annotation protocol and a subset of 5,700 manually re-annotated questions. |
| Outcome: | The proposed framework is based on 5,700 re-annotated questions from the MMLU benchmark. |
Theorem Prover as a Judge for Synthetic Data Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that large language models are increasingly capable of tackling mathematical problems. |
| Approach: | They propose an approach that iteratively refines theorem prover formalisation to mitigate errors. |
| Outcome: | The proposed method increases execution rate on the Lean prover from 60% to 87%, while human annotation is replaced with theorem prover feedback. |